Robust Tracking of Multiple Objects in Sector-Scan Sonar Image Sequences Using Optical Flow Motion Estimation

The fast update rate and good performance of new generation electronic sector scanning sonars is now allowing practicable use of temporal information for signal processing tasks such as object classification and motion estimation. Problems remain, however, as objects change appearance, merge, maneuver, move in and out of the field of view, and split due to poor segmentation. This paper presents an approach to the segmentation, two-dimensional motion estimation, and subsequent tracking of multiple objects in sequences of sector scan sonar images. Applications such as ROV obstacle avoidance, visual servoing, and underwater surveillance are relevant. Initially, static and moving objects are distinguished in the sonar image sequence using frequency-domain filtering. Optical flow calculations are then performed on moving objects with significant size to obtain magnitude and direction motion estimates. Matches of these motion estimates, and the future positions they predict, are then used as a basis for identifying corresponding objects in adjacent scans. To enhance robustness, a tracking tree is constructed storing multiple possible correspondences and cumulative confidence values obtained from successive compatibility measures. Deferred decision making is then employed to enable best estimates of object tracks to be updated as subsequent scans produce new information. The method is shown to work well, with good tracking performance when objects merge, split, and change shape. The optical flow is demonstrated to give position prediction errors of between 10 and 50 cm (1%–5% of scan range), with no violation of smoothness assumptions using sample rates between 4 and 1 frames/s.